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Abstract 


This  report  describes  an  experiment  examining  combat  identification  (CID)  decision  making 
and  the  impact  of  cue  uncertainty  on  subjects’  decision  accuracy.  Subjects  played  the  role  of 
a  dismounted  infantry  soldier  in  a  first-person  perspective  environment  and  engaged  a  series 
of  simulated  targets.  Subjects  attempted  to  engage  (i.e.,  shoot)  only  those  figures  that  were 
enemies.  Friendly  and  enemy  forces  were  distinguishable  by  differences  in  uniforms, 
equipment,  and  whether  or  not  they  were  identified  as  friendly  in  the  CID  system.  Variability 
in  the  characteristics  of  friend  and  enemy  trials  was  introduced  across  blocks.  Two  factors 
were  considered:  1)  whether  the  uncertain  characteristic  was  visual  or  behavioural,  and  2) 
whether  the  uncertain  feature  was  salient  or  not  salient.  Subjects’  hit  rates  but  not  false  alarm 
rates  were  affected  by  uncertainty  associated  with  visual  and  behavioural  characteristics  of 
targets  in  the  environment.  Subjects’  sensitivity  was  also  not  affected  by  cue  uncertainty, 
indicating  that  they  remained  equally  good  at  distinguishing  friend  and  foe  despite  variations 
in  the  characteristics  of  friends  and  foes.  Subjects  did,  however,  exhibit  differences  in 
response  bias  across  blocks.  This  result  suggests  that  subjects  shifted  their  decision  criterion 
in  response  to  cue  uncertainty  associated  with  targets.  Subjects  appear  to  have  slightly 
lowered  their  criterion  when  enemies  appeared  with  uncertain  cues  but  more  dramatically 
raised  their  criterion  when  friendly  contacts  appeared  with  uncertain  cues. 


Resume 

Le  present  rapport  decrit  une  experience  qui  porte  sur  la  prise  de  decision  dans  T  identification 
au  combat  (IDCbt),  et  qui  examine  Timpact  des  «  indices  incertains  »  sur  la  justesse  des 
decisions  prises  par  les  participants.  Dans  cette  experience,  chacun  des  participants  joue  le 
role  d’un  soldat  d’infanterie  a  pied  dans  une  perspective  de  premier  intervenant,  et  des 
objectifs  simules  (figures  humaines)  lui  sont  presentes.  II  essaie  de  ne  tirer  que  sur  les  figures 
representant  un  ennemi.  Les  forces  amies  et  ennemies  se  distinguent  par  leurs  uniformes  et 
leurs  equipements,  et  par  leur  code  d’identification  (ami  ou  ennemi)  dans  le  systeme  d’lDCbt. 
Des  caracteristiques  variables  associees  aux  amis  et  aux  ennemis  sont  introduites  dans  les 
differents  blocs  d’essais.  Deux  facteurs  sont  pris  en  consideration  :  1)  Les  indices  incertains 
sont-ils  visuels  ou  comportementaux?  2)  Les  indices  incertains  sont-ils  saillants  ou  non 
saillants?  Le  taux  de  succes  des  participants,  mais  pas  le  taux  de  fausse  alarme,  est  influence 
par  Tincertitude  associee  aux  caracteristiques  visuelles  et  comportementales  des  objectifs  dans 
l’environnement  operationnel.  La  reactivite  des  participants  n’est  pas  affectee  non  plus  par 
cette  incertitude  :  leur  capacite  de  distinguer  un  ami  d’un  ennemi  se  maintient  au  meme 
niveau  quelles  que  soient  les  variations  dans  les  caracteristiques  des  amis  et  des  ennemis. 
Cependant,  dans  les  differents  blocs  d’essais,  les  participants  presentent  des  differences  dans 
le  taux  de  reponse  erronee.  Ce  resultat  semble  indiquer  que  les  participants  ont  modifie  leurs 
criteres  de  decision  en  reponse  aux  indices  incertains  associes  aux  objectifs.  Les  participants, 
semble-t-il,  ont  abaisse  legerement  leurs  criteres  lorsque  des  soldats  ennemis  presentaient  des 
indices  incertains,  mais  les  ont  augmente  considerablement  lorsque  des  soldats  amis 
presentaient  des  indices  incertains. 
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Executive  summary 

Impact  of  Uncertain  Cues  on  Combat  Identification  Judgments 

Bryant,  D.J.,  &  Smith,  D.J.;  DRDC  Toronto  TR  2009-127;  Defence  R&D 

Canada  -  Toronto;  November  2009. 

Combat  Identification  (CID)  is  the  capability  to  identify  friendly,  enemy  and  neutral  forces 
rapidly  and  accurately,  manage  and  control  the  battlespace,  and  optimally  employ  weapons 
and  forces.  When  CID  fails,  however,  the  result  can  be  fratricide  or  neutricide,  which  have 
significant  negative  impact  on  operational  effectiveness. 

CID  is  a  complex  cognitive  as  well  as  technological  process.  Although  improvements  in 
decision  support  can  enhance  the  effectiveness  of  CID,  it  is  nevertheless  necessary  to 
understand  the  human  decision  making  processes  involved  in  identification.  As  a  first  step 
toward  building  a  more  detailed  cognitive  model  of  CID  decision  making,  we  investigated  the 
target  identification  aspect  of  CID  and  attempted  to  determine  how  human  decision  makers 
deal  with  uncertainty. 

The  experiment  described  in  this  report,  subjects  played  the  role  of  a  dismounted  infantry 
soldier  in  a  first-person  perspective  environment.  Subjects  were  presented  with  a  series  of 
trials,  in  which  each  trial  comprised  a  human  figure  moving  into  view.  The  subject’s  task  was 
to  engage  (i.e.,  shoot)  only  those  figures  that  were  enemies.  Friendly  and  enemy  forces  were 
distinguishable  by  differences  in  uniforms,  equipment,  and  behaviour.  Trials  were  grouped 
into  blocks.  In  each  block,  the  subject  remained  at  a  specific  fixed  location  in  the  simulated 
environment.  His  or  her  accuracy  and  speed  in  engaging  enemies  were  the  primary 
experimental  measures.  To  investigate  the  impact  of  cue  uncertainty  on  engagement  decision 
making  accuracy,  variability  in  the  characteristics  of  friends  and  enemies  was  introduced 
across  blocks.  Two  factors  were  manipulated:  1)  the  type  of  characteristic  that  was  uncertain 
(visual  or  behavioural),  and  2)  the  salience  of  the  uncertain  feature  (salient  or  not  salient). 

Subjects’  hit  rates  but  not  false  alarm  rates  were  affected  by  uncertainty  associated  with  visual 
and  behavioural  characteristics  of  targets  in  the  environment.  Subjects’  sensitivity  was  also 
not  affected  by  cue  uncertainty,  indicating  that  they  remained  equally  good  at  distinguishing 
friend  and  foe  despite  variations  in  the  characteristics  of  friends  and  foes.  Subjects  did, 
however,  exhibit  differences  in  response  bias  across  blocks,  suggesting  that  subjects  shifted 
their  decision  criterion  in  response  to  cue  uncertainty  associated  with  targets.  Subjects  appear 
to  have  slightly  lowered  their  criterion  when  enemies  appeared  with  uncertain  cues  but  more 
dramatically  raised  their  criterion  when  friendly  contacts  appeared  with  uncertain  cues. 

The  nature  of  modem  warfare  makes  uncertainty  a  pressing  issue.  Soldiers  can  expect  to  find 
themselves  operating  in  environments  in  which  friendly,  enemy,  and  neutral  factions  employ 
similar  or  identical  equipment,  wear  similar  or  identical  clothing,  and  potentially  employ 
similar  tactics.  Hostile  factions  in  asymmetric  conflicts  can  be  expected  to  purposely  mimic 
civilians  in  order  to  confuse  soldiers.  In  addition,  the  Canadian  Forces  (CF)  often  works  in 
coalitions,  which  can  undermine  soldiers’  familiarity  with  the  appearance  of  friendly  units. 
The  risk  posed  by  cue  uncertainty  may  be  mitigated  through  changes  to  tactics  and 
procedures,  training  and  education,  and  development  of  decision  support  systems. 
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Impact  des  indices  incertains  sur  les  jugements  humains  d’identification  au  combat 

Bryant,  D.J.,  &  Smith,  D.J.;  DRDC  Toronto  TR  2009-127;  Defence  R&D 

Canada  -  Toronto;  novembre  2009. 

L’ identification  au  combat  (IDCbt)  est  la  capacite  d’ identifier  les  forces  amies,  ennemies  et 
neutres  de  fagon  rapide  et  precise,  de  gerer  et  controler  l’espace  de  bataille,  et  d’employer  les 
armes  et  les  forces  de  fagon  optimale.  Cependant,  lorsque  1’lDCbt  echoue,  cela  peut  donner 
lieu  a  un  incident  fratricide  ou  neutricide  qui  a  un  impact  negatif  considerable  sur  l’efficacite 
operationnelle. 

LTDCbt  est  un  processus  cognitif  et  technologique  complexe.  Les  ameliorations  apportees 
aux  outils  d’aide  a  la  decision  peuvent  ameliorer  l’efficacite  de  1’IDCbt,  mais  il  demeure 
necessaire  de  comprendre  les  processus  de  prise  de  decision  qui  entrent  en  jeu  dans 
f  identification.  Comme  premiere  etape  dans  l’elaboration  d’un  modele  cognitif  plus  detaille 
de  la  prise  de  decision  dans  1’lDCbt,  nous  avons  examine  l’aspect  «  identification  des 
objectifs  »  de  1’IDCbt,  etnous  avons  essaye  de  determiner  comment  les  decideurs  humains 
font  face  a  1’ incertitude. 

Dans  l’experience  que  decrit  le  present  rapport,  chacun  des  participants  joue  le  role  d’un 
soldat  d’infanterie  a  pied  dans  une  perspective  de  premier  intervenant.  L’experience  consiste 
en  une  serie  d’essais  dans  lesquels  une  figure  humaine  est  exposee  soudainement  a  la  vue  du 
participant.  Celui-ci  ne  doit  tirer  que  sur  les  figures  representant  un  ennemi.  Les  forces  amies 
et  ennemies  se  distinguent  par  leurs  uniformes,  leurs  equipements  et  leur  comportement.  Les 
essais  sont  groupes  en  blocs.  Dans  chaque  bloc,  le  participant  demeure  a  un  endroit  fixe  dans 
l’environnement  simule.  La  precision  et  la  vitesse  avec  laquelle  il  tire  sur  1’ ennemi  sont  les 
principaux  criteres  de  mesure  experimentale.  Pour  mesurer  L impact  des  «  indices  incertains  » 
sur  la  justesse  des  decisions  prises  par  les  participants,  des  caracteristiques  variables  associees 
aux  amis  et  aux  ennemis  sont  introduites  dans  les  differents  blocs  d’essais.  Deux  facteurs  sont 
pris  en  consideration  :  1)  Les  indices  incertains  sont-ils  visuels  ou  comportementaux?  2)  Les 
indices  incertains  sont-ils  saillants  ou  non  saillants? 

Le  taux  de  succes  des  participants,  mais  pas  le  taux  de  fausse  alarme,  est  influence  par 
1’ incertitude  associee  aux  caracteristiques  visuelles  et  comportementales  des  objectifs  dans 
l’environnement  operationnel.  La  reactivite  des  participants  n’est  pas  affectee  non  plus  par 
cette  incertitude  :  leur  capacite  de  distinguer  un  ami  d’un  ennemi  se  maintient  au  meme 
niveau  quelles  que  soient  les  variations  dans  les  caracteristiques  des  amis  et  des  ennemis. 
Cependant,  dans  les  differents  blocs  d’essais,  les  participants  presentent  des  differences  dans 
le  taux  de  reponse  erronee.  Ce  resultat  semble  indiquer  que  les  participants  ont  modifie  leurs 
criteres  de  decision  en  reponse  aux  indices  incertains  associes  aux  objectifs.  Les  participants, 
semble-t-il,  ont  abaisse  legerement  leurs  criteres  lorsque  des  soldats  ennemis  presentaient  des 
indices  incertains,  mais  les  ont  augmente  considerablement  lorsque  des  soldats  amis 
presentaient  des  indices  incertains. 

Dans  la  guerre  modeme,  l’incertitude  un  probleme  tres  preoccupant.  Les  soldats  peuvent  se 
retrouver  dans  un  environnement  ou  les  factions  amies,  ennemies  et  neutres  utilisent  des 
equipements  similaires  ou  identiques,  portent  des  uniformes  similaires  ou  identiques,  et 
emploient  des  tactiques  similaires.  Dans  les  conflits  asymetriques,  il  faut  s’attendre  a  ce  que 
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les  ennemis  imitent  le  comportement  des  civils  pour  jeter  la  confusion  parmi  les  soldats.  De 
plus,  les  Forces  canadiennes  (FC)  interviennent  souvent  dans  le  cadre  d’une  coalition,  ce  qui 
peut  nuire  a  la  capacite  de  nos  soldats  a  reconnaitre  les  unites  amies.  Le  risque  lie  aux  indices 
incertains  peut  etre  attenue  par  la  modification  des  tactiques,  des  procedures  et  des 
programmes  destruction  et  d’education,  et  par  le  developpement  de  systemes  d’aide  a  la 
decision. 
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Introduction 


Combat  Identification  (CID)  is  the  capability  to  identify  friendly,  enemy  and  neutral  forces 
rapidly  and  accurately,  manage  and  control  the  battlespace,  and  optimally  employ  weapons  and 
forces  [1],  The  goal  of  the  decision  maker  in  CID  is  to  ensure  that  when  a  weapon  is  fired,  it  is 
fired  at  an  appropriate  target.  Therefore,  CID  involves  rapidly  and  accurately  identifying  the 
allegiance  (e.g.,  friend,  enemy,  neutral)  of  contacts  detected  in  the  battlespace,  based  on  all 
available  sources  of  data.  Ultimately,  the  purpose  of  CID  is  to  gain  the  maximum  combat 
effectiveness  possible,  which  involves  minimizing  losses  caused  by  both  enemy  and  friendly  fire 
[2], 

In  most  operational  settings,  numerous  sources  of  data  are  available  and  must  be  considered  to 
correctly  identify  targets.  These  data  sources  include  surveillance,  navigation,  and  networking. 
CID  is  generally  considered  to  entail  three  elements:  situation  awareness  (SA),  target 
identification,  and  Tactics,  Techniques,  and  Procedures  (TTPs)  (e.g.,  Dean  &  Handley  [2]).  SA 
refers  to  the  perception  and  understanding  of  the  operational  environment  needed  to  act 
effectively  in  that  environment.  CID  clearly  requires  SA  as  a  precursor  to  the  classification  of 
entities  as  friendly,  hostile,  or  neutral.  Target  identification  is  the  process  of  making  that 
classification  judgment  based  on  the  characteristics  of  the  entity  in  question  in  relation  to  the 
TTPs  that  govern  how  one  inteiprets  objects  in  the  operational  environment.  One  can  think  of  SA 
as  providing  the  data  about  objects  in  the  environment  and  TTPs  providing  the  knowledge  needed 
to  interpret  that  data.  Target  identification  is  thus  the  process  by  which  SA  and  TTPs  are 
employed  [2], 

Another  view  of  CID  focuses  on  the  decision  making  steps  required.  Dean  et  al.  [3]  propose  a 
four-stage  model  of  CID  comprised  of: 

•  Detection:  Determining  that  a  target  of  potential  interest  is  present; 

•  Classification:  Assigning  the  target  to  a  general  class  of  object  or  entity,  such  as  an 
armoured  vehicle,  tank,  etc.; 

•  Identification:  Determining  the  allegiance  of  the  target  (friend,  enemy,  neutral);  and 

•  Action:  Determining  the  appropriate  action  to  take  with  respect  to  the  target,  based  on 
Rules  of  Engagement  (ROE)  and  Standard  Operating  Procedures  (SOP). 

Why  is  CID  Important? 

CID  is  the  process  by  which  enemies  are  identified  and  targeted  for  destruction  and  is  a  key 
element  of  combat  effectiveness.  It  is  generally  the  failures  of  CID  that  cause  the  most  concern 
both  operationally  and  nationally.  Failure  of  a  CID  system  can  lead  to  fratricide  (the 
inappropriate  engagement  of  a  friendly  soldier  or  unit),  neutricide1  (identifying  a  neutral  contact 
as  hostile),  or  injury  or  death  to  oneself  caused  by  failure  to  identify  an  enemy  contact2  (see 
Harris  &  Syms  [4]). 


1  Neutricide  is  the  term  used  by  Dean  et  al.  [3]  to  describe  incidents  when  civilians  and  civilian  infrastructure  are  accidentally 
targeted  or  misidentified  and  deliberately  targeted. 

2  Referred  to  as  a  mistake  akin  to  ‘suicide’  on  the  battlefield  by  Karsh,  Walrath,  Swoboda  and  Pillalamarri  [5], 
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Causing  accidental  death  or  injury  is  only  one  of  the  negative  effects  of  fratricide  and  neutricide. 
Other  negative  effects  are  listed  in  Table  1.  Some  affect  the  combat  effectiveness  of  one’s  own 
forces,  as  fratricide  can  lead  to  loss  of  morale  as  well  as  a  reluctance  to  take  risk  and  seize  the 
initiative.  Other  effects  are  more  systemic,  changing  the  nature  of  command  and  control  or 
causing  political  or  legal  issues.  Neutricide  can  likewise  create  political  and  legal  issues  and 
undermine  efforts  to  win  over  local  civilian  populations. 


Table  1:  Negative  Effects  of  Fratricide  and  Neutricide 

Source:  Dean  et  al.  (2005)  [3] 

Source:  U.S.  Congress,  Office  of  Technology 
Assessment 

Casualties  &  damage  to  equipment 

Hesitation  to  conduct  limited  visibility  operations 

Wasted  time,  effort,  and  ammunition 

Loss  of  confidence  in  leadership 

Drop  in  morale  and  levels  of  trust 

Increase  of  leader  self-doubt 

Drop  in  unit  effectiveness  and  excessive  caution 

Hesitation  to  use  supporting  combat  systems 

Strain  on  coalitions  if  casualties  inflicted  on  allies 

Oversupervision  of  units 

Political  repercussions 

Loss  of  initiative 

Loss  of  “hearts  and  minds”  of  civilian  population 

Loss  of  aggressiveness  during  fire  and  maneuver 

Unnecessary  risk  to  own  forces 

Disrupted  operations 

Disruption  of  tempo 

Needless  loss  of  combat  power 

General  degradation  of  cohesion  and  morale 

How  prevalent  are  failures  of  CID? 

Estimating  the  frequency  of  fratricide  has  historically  been  a  difficult  task.  Generally,  little 
reliable  evidence  is  available  and  the  tempo  of  warfare  works  against  the  preservation  of  sites 
where  fratricide  may  have  occurred  [6].  Estimates  from  the  First  and  Second  World  Wars  put  the 
percentage  of  all  Allied  casualties  caused  by  friendly  fire  between  10  and  15%  [3].  Harris  and 
Syms  [4]  conducted  a  thorough  review  of  historical  reports  and  documents  that  indicated  similar 
fratricide  rates  for  those  conflicts  but  suggested  higher  rates  for  more  recent  conflicts.  In  line 
with  this,  nearly  80%  of  casualties  suffered  by  the  United  Kingdom  in  Operation  Granby  (1st  Iraq 
war)  were  attributed  to  friendly  fire  [3].  The  United  States  estimated  a  much  lower  fratricide  rate 
in  that  conflict  but  one  still  higher  than  those  estimated  in  earlier  wars. 

Several  factors  potentially  contribute  to  a  greater  risk  of  fratricide  in  the  modern  battlespace. 
Weapons  have  much  longer  ranges  than  in  past  conflicts,  and  targets  can  be  engaged  before  it  is 
possible  to  acquire  positive  identification  [7],  This  creates  a  dangerous  situation  in  which  forces 
must  be  concerned  that  a  target  is  a  potential  enemy  who  could  fire  upon  them  but  are  unable  to 
gather  the  data  necessary  for  CID.  The  greater  range  of  weapon  systems  also  means  that  remote 
sensors,  which  may  provide  only  partial  cues  to  identity,  must  be  relied  on  to  a  greater  extent. 
Greater  mobility  of  forces  has  led  to  operational  environments  in  which  forces  are  more 
dispersed,  making  it  more  difficult  to  maintain  good  SA  [1]. 

In  addition  to  the  greater  risk  of  fratricide  and  neutricide,  the  use  of  modern  weapons  and 
surveillance  allows  fratricides  and  neutricides  to  be  more  easily  detected  [1].  Hence  accidental 
deaths  and  injuries  that  may  have  gone  unexplained  in  past  conflicts  may  now  be  more  accurately 
detected. 
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Perhaps  the  most  significant  factor  working  against  CID  in  today’s  environment  is  the 
increasingly  asymmetric  nature  of  conflict.  Canada  and  its  coalition  partners  find  themselves 
participating  in  high  tempo,  non-linear  operations  with  enemies  who  eschew  traditional  uniforms 
and  employ  diverse  equipment.  The  presence  of  civilians  further  complicates  the  environment. 


What  causes  CID  failures? 

There  is  no  single  cause  for  incidents  of  fratricide  and  neutricide.  Major  risk  factors  are  the  loss 
of  SA  and  misidentification  of  the  target  [4]  [7]  [8].  Each  of  these  factors,  however,  is  a 
confluence  of  more  proximal  factors  that  break  down  further  into  human,  physical,  and 
organizational  factors  [2]. 

Human  factors  are  characteristics  or  traits  of  human  beings,  related  to  their  physiology,  cognitive 
capabilities,  and  development  (e.g.,  through  training),  that  can  negatively  affect  CID 
performance.  For  example,  people  have  natural  limits  to  information  processing  capacity  [9] 
which  makes  it  difficult  for  soldiers  to  maintain  SA  in  complex  environments  [7].  CID  is  made 
especially  difficult  for  soldiers  in  environments  such  as  Afghanistan  by  the  asymmetric  nature  of 
that  conflict,  characterized  by  a  difficulty  in  knowing  who  and  where  one’s  enemies  are  and  how 
and  when  they  will  attack.  Human  beings  are  also  subject  to  stress  and  emotions  that  can  impair 
performance,  leading  to  misidentifications,  lack  of  fire  discipline,  etc.  Training  and  education 
can  be  positive  factors  but  poor  training  can  impair  both  SA  and  identification  [2]. 

Physical  factors  include  environmental  conditions  and  the  state  of  equipment,  especially  sensors. 
Environmental  conditions  that  reduce  visibility  or  hinder  the  functioning  of  sensors  are  key 
factors  in  many  fratricide  incidents  [10].  Equipment  failures  can  also  make  CID  more  difficult 
and  error-prone.  Increasingly,  operational  zones  feature  the  presence  of  similar  or  even  identical 
equipment  being  used  by  friendly,  neutral,  and  enemy  forces  and  this  can  cause  tremendous 
confusion  in  the  identification  process  [7]. 

Operational  factors  pertain  to  the  unique  geographical,  cultural,  and  historic  features  of  the 
operational  setting,  as  well  as  the  organizational  structure  in  which  soldiers  function.  Operating 
afield  in  unfamiliar  nations  can  leave  soldiers  with  limited  knowledge  about  data  sources  needed 
to  distinguish  neutral  from  potentially  hostile  factions  [11].  It  is  often  the  case  that  such 
knowledge  is  difficult  and  time-consuming  to  acquire.  Constraints  imposed  from  higher 
command  in  the  form  of  SOPs  and  ROE  can  further  hinder  the  CID  process.  Failures  of 
command  and  control  (C2)  and  communication  frequently  contribute  to  fratricide  and  neutricide 
incidents  [10].  All  of  these  issues  are  exacerbated  in  high-tempo  operations  that  decrease 
margins  of  error  [4]. 


How  can  CID  be  Improved? 

CID  performance  can  be  improved  by  addressing  each  of  its  elements:  a)  target  identification,  b) 
SA,  and  c)  TTPs  [2],  The  US  Armed  Forces,  for  example,  has  developed  concepts  for  decision 
support  to  both  SA  and  target  identification  (see  [1]  for  a  review).  In  the  former  case,  it  has 
considered  so-called  “blue-force  tracking”  systems  that  track  the  positions  of  all  friendly  vehicles 
and  transmits  this  information.  Significant  work  in  this  area  has  been  done  by  the  U.S.  Joint 
Force  Command’s  (USJFCOM)  Coalition  Combat  Identification  (CCID)  team  through  its 
Advanced  Concept  Technology  Demonstrations  (ACTDs).  In  addition,  the  US  Armed  Forces  has 
employed  visual  markings,  radio  emission  intercept,  and  IFF  (Identify  Friend/Foe)  systems  to 
assist  in  target  identification  [5]  [7]. 
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Although  Canada  is  participating  in  efforts  to  develop  SA  and  target  identification  decision 
support  tools,  the  Canadian  Forces  (CF)  has  also  worked  on  refining  TTPs  to  enhance  CID  [12]. 
This  work  includes  the  development  of  enhanced  training  as  well  as  new  SOPs. 


Understanding  Human  Judgment  in  CID 

CID  is  a  complex  cognitive  as  well  as  technological  process.  Although  improvements  in  sensors 
and  development  of  Blue  Force  Tracking  (BFT)  devices3  can  enhance  the  effectiveness  of  CID,  it 
is  nevertheless  necessary  to  understand  the  human  decision  making  processes  involved  in 
identification.  The  nature  of  the  decision  making  process  determines  how  human  factors  affect 
CID  and  can  also  help  us  better  understand  the  kinds  of  physical  and  operational  conditions  that 
will  challenge  soldiers. 

One  such  CID  model  is  the  INtegrative  Combat  IDentification  Entity  Relationship  (INCIDER) 
model  [2]  [3],  This  model  integrates  the  effects  of  physical,  operational,  and  human  factors  in  a 
description  of  the  steps  required  to  identify  a  target.  In  brief,  the  model  assumes  that  the  human 
decision  maker  fuses  multiple  sensory  inputs  and  SA,  as  inteipreted  with  respect  to  his/her 
knowledge  base,  to  produce  a  classification  of  the  target.  The  decision  maker  assesses  his/her 
confidence  in  that  classification  based  on  various  parameters  and  expectations  and  generates  a 
decision  on  what  action  to  take.  Although  the  INCIDER  model  offers  a  framework  in  which  to 
understand  CID,  it  is  not  very  specific  concerning  the  nature  of  the  cognitive  process  by  which 
the  decision  maker  fuses  data,  classifies  the  target,  and  determines  the  appropriate  response. 

Dean  et  al.  [3]  assume  that  people  employ  a  recognition-based  process  along  the  lines  of  Klein’s 
[13]  Recognition-Primed  Decision  (RPD)  model  to  recognize  targets.  The  RPD  model  does 
emphasize  the  role  of  SA  in  decision  making  and,  as  such,  seems  consistent  with  what  is  known 
about  CID.  However,  like  the  INCIDER  model,  the  RPD  model  also  does  not  provide  a  detailed 
process-oriented  model  with  which  to  predict  the  effects  of  human  factors  on  CID  judgments  and 
decision  support. 

In  this  report,  we  focus  on  the  target  identification  aspect  of  CID  and  attempt  to  determine  how 
human  decision  makers  deal  with  uncertainty.  The  modern  battlespace  presents  instances  in 
which  the  physical  characteristics  of  friend,  neutral,  and  enemy  may  be  similar  or  identical  [7]. 
Soldiers  are  therefore  faced  with  the  difficult  task  of  determining  identity  based  on  cues  that  are 
only  imperfectly  associated  with  a  target  class.  As  a  first  step  in  building  a  more  detailed 
cognitive  model  of  CID  decision  making,  we  investigated  the  process  by  which  people  aggregate 
and  fuse  multiple  data  sources,  or  cues,  to  identify  targets. 

The  experiment  described  in  this  report  explores  elements  of  human  CID  decision  making.  In 
particular,  the  experiment  provides  insight  into  the  way  people  deal  with  the  uncertainty 
associated  with  various  kinds  of  cues  (visual  and  behavioural)  to  indicate  whether  a  target  is 
friendly  or  hostile.  This  will  help  us  determine  plausible  models  of  human  information 
aggregation  for  CID  (see  Famewo,  Matthews,  &  Lamoureux  [14]).  By  identifying  plausible 
models  of  human  behaviour,  we  will  be  able  to  support  development  of  procedures,  training,  and 
technology  to  enhance  combat  ID  decision  making. 


3  A  BFT  is  a  system  that  collects  positional  information  from  friendly  units  carrying  appropriate 
transmitters,  collates  this  information,  and  distributes  a  composite  map  of  friendly  unit  positions. 
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Experiment 


This  experiment  was  conducted  using  the  IMMERSIVE  (Instrumented  Military  Modeling  Engine 
for  Research  using  Simulation  and  Virtual  Environments)  software  platform  developed  at 
Defence  Research  &  Development  Canada  (DRDC)  Valcartier.  IMMERSIVE  is  based  on  a 
modified  gaming  environment  called  “Unreal  Tournament”  that  creates  a  first-person  perspective 
environment  in  which  the  subject  assumes  the  role  of  a  dismounted  infantry  soldier.  This 
platform  was  used  to  present  subjects  with  a  series  of  blocks  of  trials,  in  which  each  trial 
comprised  a  human  figure  moving  into  view.  The  subject’s  task  was  to  engage  (i.e.,  shoot)  only 
those  figures  that  were  enemies.  Friendly  and  enemy  forces  were  distinguishable  by  differences 
in  uniforms,  equipment,  and  behaviour.  In  each  block,  subjects  remained  at  a  specific  fixed 
location  in  the  simulated  environment.  Their  accuracy  in  engaging  enemies  was  the  primary 
experimental  measure. 

The  initial  objective  of  the  experiment  was  to  investigate  the  impact  of  cue  uncertainty  on 
engagement  decision  making  performance.  To  examine  this  objective,  variability  in  the 
characteristics  of  friend  and  enemy  trials  was  introduced  across  blocks.  Two  factors  were 
considered:  1)  the  type  of  characteristic  that  was  uncertain  (visual  or  behavioural),  and  2)  the 
salience  of  the  uncertain  feature  (salient  or  not  salient).  These  factors  were  systematically  varied 
across  blocks.  Visual  characteristics  are  perhaps  of  primary  importance  to  dismounted  soldiers, 
but  the  behaviour  of  individuals  provides  valuable  information  as  well  [15]. 


Method 

Subjects 

Subjects  were  30  male  and  female  employees  of  DRDC  Toronto,  students  conducting  research  at 
DRDC  Toronto,  or  individuals  recruited  from  local  universities.  All  subjects  were  aged  18  and 
older,  had  normal  or  corrected-to-normal  vision,  and  were  unfamiliar  with  the  specific  hypotheses 
and  stimulus  configurations  of  the  experiments.  All  received  stress  pay  remuneration  for 
participating. 

This  study,  approved  by  the  DRDC  Toronto  Human  Research  Ethics  Committee  (HREC)  and  was 
conducted  in  conformity  with  the  Tri-Council  Policy  Statement:  Ethical  Conduct  for  Research 
Involving  Humans. 


Materials 

The  experiment  was  conducted  with  Personal  Computers  (PCs),  which  presented  stimuli, 
collected  subject  responses,  and  recorded  data.  The  IMMERSIVE  platform  was  used  as  a  test 
bed  simulating  combat  activities. 

The  experimentation  process  comprised  the  set  up,  deployment,  and  management  of  the  following 
components: 

•  Terrain:  The  simulated  environment  in  which  a  scenario  takes  place; 

•  Scenario:  A  sequence  of  events  representing  a  portion  of  battlefield  action; 
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•  roBOTic  computer  controlled  entities  (BOTs):  Play  scenario  characters  (see  Figure  1); 

•  Subject:  Plays  the  role  of  a  Canadian  soldier  and  controls  a  rifle  that  can  be  used  to 
engage  (shoot  at)  hostile  entities;  and 

•  Rules  of  engagement:  Rules  that  govern  how  the  subject  responds  to  different  kinds  of 
BOTs. 


Figure  1.  Examples  of  BOTs  to  be  used  in  scenarios  (pictures  presented  here  are  not  as  sharp  as 

actual  computer  images). 


Subjects  were  provided  with  ROE  at  the  beginning  of  the  experimental  session.  The  ROE 
distinguished  friendly  from  potentially  hostile  BOTs  and  governed  when  the  subject  was  to 
engage  hostile  BOTs  with  the  rifle. 

Each  subject  perfonned  10  blocks  of  trials  as  a  dismounted  infantry  soldier  in  a  three-dimensional 
urban  environment.  Subjects  interacted  with  this  environment  from  a  fixed  location  (indicated  by 
the  white  box  in  Figure  2),  giving  them  a  viewpoint  within  the  environment.  Note,  subjects  never 
saw  a  top-down  view  as  illustrated  in  Figure  2.  In  a  block,  a  number  of  computer-controlled 
BOTs  were  in  motion,  following  pre-specified  paths  at  pre-specified  times.  Two  such  paths  are 
marked  in  blue  and  red  in  Figure  2.  The  BOTs  traveled  into  and  out  of  view  sequentially,  so  that 
no  two  BOTs  were  visible  to  the  subject  at  the  same  time. 

The  Combat  ID  Virtual  Simulation  software  logged  subject  actions  pertinent  to  subjects’  firing 
decisions.  The  software  logged  each  instance  in  which  the  subject  fired  the  rifle,  the  identity  of 
the  BOT  fired  upon,  and  the  subject’s  accuracy  (whether  or  not  the  shot  hit  the  BOT).  The 
primary  experimental  measure  was  decision  accuracy  (i.e.,  whether  or  not  the  subject  engaged  a 
hostile  or  friendly  BOT).  A  subject  could  fire  one  or  more  shots  at  a  BOT  without  hitting  it.  To 
capture  such  events  as  intended  engagements,  the  software  logged  for  each  shot  fired  whether  it 
hit  a  BOT  and,  if  not,  how  close  the  shot  was  to  a  BOT  (i.e.  minimum  distance  between  shot  and 
BOT).  Shots  fired  within  a  certain  distance  of  a  BOT  (approximately  one  meter  in  the  simulated 
environment)  were  counted  as  engagements.4 


4  Because  the  criterion  for  engagement  was  a  fixed  distance  with  respect  to  a  BOT,  the  angular 
displacement  from  rifle  to  BOT  varied  somewhat  with  the  distance  of  the  BOT  to  the  subject’s  firing 
position.  However,  angular  displacements  were  not  computed  for  each  shot  because  the  range  of  firing 
angles  associated  with  engagements  was  fairly  small. 
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Design 

The  initial  objective  of  the  experiment  was  to  investigate  the  impact  of  cue  uncertainty  on 
engagement  decision  making  (accuracy,  speed).  Two  factors  were  manipulated:  1)  the  BOT 
characteristic  that  was  uncertain  (visual  or  behavioural),  and  2)  the  salience  of  the  uncertain 
feature  (salient  or  not  salient).  These  factors  were  manipulated  within-subjects,  across  blocks. 


Figure  2.  Example  BOT  paths  in  a  simulated  environment. 


Friendly  and  hostile  BOTs  were  designed  on  the  basis  of  a  common  model  that  could  be  varied 
according  to  two  visual  and  two  behavioural  features  (see  Table  2).  For  both  visual  and 
behavioural  characteristics,  one  feature  was  relatively  salient  and  the  other  less  so.  BOTs 
designated  friendly  or  hostile  differed  only  along  these  four  characteristics. 
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The  salient  visual  characteristic  was  the  color  of  the  BOT’s  uniform.  The  standard  friendly  value 
was  the  green  Canadian  Distinctive  Pattern  (CADPAT)  used  by  the  CF,  whereas  the  enemy  value 
was  a  distinct  olive  colour.  The  less  salient  visual  characteristic  was  the  rifle  carried  by  a  BOT, 
which  was  either  a  C7  rifle  (friend)  or  AK-47  (enemy).  Salience  of  visual  cues  was  based  on  the 
relative  size  of  the  uniform  and  rifle,  with  the  larger  uniform  being  judged  more  salient  than  the 
rifle.  The  salience  of  behavioural  characteristics  was  based  on  the  ease  with  which  a  subject 
could  notice  that  characteristic.  The  salient  behavioural  characteristic  was  the  direction  in  which 
a  BOT  moved  when  it  appeared  in  the  subject’s  field  of  view.  Friendly  BOTs  moved  away  from 
the  subject’s  position,  whereas  enemy  BOTs  moved  toward  the  subject’s  position.  The  low- 
salience  behavioural  characteristic  was  the  presence  or  absence  of  a  pause  in  the  BOT’s  motion. 
Friendly  BOTs  paused  for  1  -  2  seconds  during  movement,  whereas  enemy  BOTs  did  not  pause  at 
any  point. 


Table  2.  Critical  BOT  Cues 

Bot  Class 

Feature  Type 

Saliency 

Feature 

Standard  Value 

Friend 

Visual 

High 

Color 

CADPAT 

Low 

Rifle 

C7 

Behavioural 

High 

Movement 

Away 

Low 

Pause 

Pause 

Enemy 

Visual 

High 

Color 

Olive 

Low 

Rifle 

AK-47 

Behavioural 

High 

Movement 

Toward 

Low 

Pause 

No  pause 

Procedure 

Subjects  were  first  briefed  on  the  general  purpose  of  the  experiment  and  their  role.  They  were 
told  that  they  would  be  engaging  in  simulated  combat  using  a  first-person-shooter  computer  game 
which  has  been  modified  for  data  collection  purposes. 

In  the  experiment  session,  subjects  played  the  role  of  a  dismounted  soldier,  which  was 
represented  in  the  simulated  environment  by  an  avatar  (a  controllable  character  in  the  simulated 
environment).  The  avatar  could  only  remain  at  its  fixed  location  and  the  subject  controlled  the 
angular  direction  of  the  avatar,  which  could  face  any  direction  in  a  180°  arc.  The  subject’s  task 
was  to  monitor  the  area  in  front  of  the  avatar’s  position  and  engage  any  and  all  enemy  soldiers 
that  moved  into  view.  Subjects  controlled  the  movement  of  the  rifle  with  the  computer  mouse 
and  fired  by  pressing  the  left  mouse  button.  Subjects  were  told  that  roughly  equal  numbers  of 
friendly  and  enemy  soldiers  would  move  through  their  area  of  responsibility,  but  that  friends  and 
enemies  could  be  distinguished  by  observing  four  characteristics  of  the  soldier. 

Subjects  were  guided  through  a  practice  session  in  which  they  were  told  how  to  control  the 
avatar,  shoot  the  rifle,  etc.  During  the  practice  session,  subjects  saw  examples  of  friendly  and 
enemy  soldiers  and  practiced  firing  the  rifle. 

In  the  main  session,  subjects  performed  a  series  of  10  blocks.  Each  block  consisted  of  20  BOTs 
(10  friend  and  10  enemy)  moving  through  the  environment.  The  starting  points  and  timings  of 
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Table  3.  Schedule  of  Blocks  and  BOT  Features 

Friend  Features 

Enemy  Features 

Block 

Name 

BOTs 

Uniform 

Rifle 

Movement 

Pause 

10  Friend 

CAD PAT 

C7 

Away 

No 

1 

Baseline 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

8  Friend 

CAD PAT 

C7 

Away 

No 

2 

Friend  Salient  Visual 
Variable 

2  Friend 

Olive* 

C7 

Away 

No 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

8  Friend 

CAD PAT 

C7 

Away 

No 

3 

Friend  Non-Salient 
Visual  Variable 

2  Friend 

CADPAT 

AK-47* 

Away 

No 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

10  Friend 

CADPAT 

C7 

Away 

No 

4 

Enemy  Salient  Visual 
Variable 

8  Enemy 

Olive 

AK-47 

Toward 

Yes 

2  Enemy 

CADPAT* 

AK-47 

Toward 

Yes 

10  Friend 

CADPAT 

C7 

Away 

No 

5 

Enemy  Non-Salient 
Visual  Variable 

8  Enemy 

Olive 

AK-47 

Toward 

Yes 

2  Enemy 

Olive 

C7* 

Toward 

Yes 

8  Friend 

CADPAT 

C7 

Away 

No 

6 

Friend  Salient 
Behavioural  Variable 

2  Friend 

CADPAT 

C7 

Toward* 

No 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

8  Friend 

CADPAT 

C7 

Away 

No 

7 

Friend  Non-Salient 
Behavioural  Variable 

2  Friend 

CADPAT 

C7 

Away 

Pause* 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

10  Friend 

CADPAT 

C7 

Away 

No 

8 

Enemy  Salient 
Behavioural  Variable 

8  Enemy 

Olive 

AK-47 

Toward 

Yes 

2  Enemy 

Olive 

AK-47 

Away* 

Yes 

10  Friend 

CADPAT 

C7 

Away 

No 

9 

Enemy  Non-Salient 
Behavioural  Variable 

8  Enemy 

Olive 

AK-47 

Toward 

Yes 

2  Enemy 

Olive 

AK-47 

Toward 

No* 

10  Friend 

CADPAT 

C7 

Away 

No 

10 

End  Baseline 

10  Enemy 

Olive 

AK-47 

Toward 

Yes 

*  Non-standard  features  are  highlighted;  except  in  baseline  conditions,  two  friend  or  enemy  BOTs  possessed  a  single 

non-standard  feature  and  all  other  BOTs  possessed  exclusively  standard  features. 

DRDC  Toronto  TR  2009-127 


9 


the  BOTs  was  varied  systematically,  so  as  to  be  unpredictable  to  subjects,  but  no  two  BOTs 
appeared  within  the  subject’s  field  of  view  at  the  same  time.  To  ensure  that  starting  position  did 
not  affect  overall  performance,  one  friend  and  one  enemy  shared  the  same  starting  point  (at 
different  times)  in  each  block. 

Table  3  indicates  for  each  block  the  features  associated  with  friend  and  enemy  BOTs.  The  first 
block  served  to  measure  subjects’  baseline  performance  and  contained  10  friends  and  10  enemies 
comprised  of  the  standard  features  associated  with  each  type  of  BOT  (see  Table  2).  In 
subsequent  blocks,  one  cue  of  either  friend  or  enemy  BOTs  was  varied  such  that  two  of  the  10 
instances  contained  non-standard  features  (i.e.,  features  associated  with  the  other  class  of  BOT). 
For  example,  in  Block  2,  eight  friendly  BOTs  possessed  all  four  features  standard  to  the 
definition  of  friend  in  the  ROE  but  for  two  friendly  BOTs  the  salient  visual  feature  associated 
with  friend  was  replaced  with  the  feature  normally  associated  with  enemy  BOTs.  All  10  enemy 
BOTs  in  Block  2  possessed  all  the  features  standard  to  the  definition  of  enemy.  As  shown  in 
Table  3,  subsequent  blocks  involved  the  switching  of  one  visual  or  behavioural  feature  at  a  time 
across  friendly  and  enemy  BOTs  to  systematically  introduce  uncertainty  as  to  what  features  a 
friend  or  enemy  could  possess.  This  was  done  according  to  the  schedule  shown  in  Table  3.  All 
subjects  followed  the  same  order  of  blocks  as  indicated  in  Table  3.  Each  row  in  Table  3  indicates 
a  block  and  the  number  of  friends  and  enemies  presented  with  the  visual  and  behavioural  features 
indicated.  The  final  block  was  the  same  as  the  first  and  served  as  a  comparison  to  the  baseline  for 
assessing  learning  effects  over  the  course  of  the  experiment. 

The  trial  block  is  a  key  factor  for  analysis  of  the  data.  To  facilitate  description  of  the  results,  the 
acronyms  listed  in  Table  4  were  used  to  refer  to  each  block.  A  brief  description  of  each  condition 
is  also  provided  in  Table  4  for  use  as  a  reference. 


Table  4:  Description  of  Experiment  Blocks 

Block 

Acronym 

Description 

Baseline 

Baseline 

Assessment  of  initial  performance  level  with  all  standard 
Friends  and  Enemies 

Friend  Salient  Visual  Variable 

FSVV 

Two  Friends  possess  a  salient  non-standard  visual 
characteristic 

Friend  Non-Salient  Visual  Variable 

FNSVV 

Two  Friends  possess  a  non-salient  non-standard  visual 
characteristic 

Enemy  Salient  Visual  Variable 

ESVV 

Two  Enemies  possess  a  salient  non-standard  visual 
characteristic 

Enemy  Non-Salient  Visual  Variable 

ENSVV 

Two  Enemies  possess  a  non-salient  non-standard  visual 
characteristic 

Friend  Salient  Behavioural  Variable 

FSBV 

Two  Friends  exhibit  a  salient  non-standard  behavioural 
characteristic 

Friend  Non-Salient  Behavioural  Variable 

FNSBV 

Two  Friends  exhibit  a  non-salient  non-standard 
behavioural  characteristic 

Enemy  Salient  Behavioural  Variable 

ESBV 

Two  Enemies  exhibit  a  salient  non-standard  behavioural 
characteristic 

Enemy  Non-Salient  Behavioural  Variable 

ENSBV 

Two  Enemies  exhibit  a  non-salient  non-standard 
behavioural  characteristic 

End  Baseline 

End  Baseline 

Final  assessment  of  performance  level  with  all  standard 
Friends  and  Enemies  (to  ascertain  learning  effects) 

Results 

Subjects’  decisions  to  shoot,  as  indicated  by  firing  a  shot  that  hit  or  came  within  a  critical  distance 
(see  Materials  section,  p.  16)  of  the  BOT,  or  to  not  shoot,  as  indicated  by  no  shot  or  a  shot  outside 
the  vicinity  of  the  BOT,  were  recorded  for  each  BOT  in  each  block.  Decisions  to  shoot  an  enemy 
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BOT  were  termed  hits  and  indicated  correct  recognition  of  an  enemy,  whereas  decisions  to  not 
shoot  an  enemy  BOT  were  termed  misses  and  indicated  failures  to  recognize  an  enemy  leading  to 
reduced  mission  effectiveness.  Decisions  to  not  shoot  friendly  BOTs  were  termed  correct 
rejections  and  comprised  the  correct  recognition  of  a  friend,  whereas  decisions  to  shoot  a  friendly 
BOT  were  termed  false  alarms  (FA)  and  indicated  the  incorrect  determination  of  an  enemy, 
leading  to  an  instance  of  fratricide.  Flit  and  false  alarm  rates  were  of  greatest  interest  in  this 
experiment. 


Hit  Data 

Figure  3  shows  the  hit  rates  calculated  for  each  block.  Overall,  hit  rates  were  relatively  high, 
exceeding  80%  in  all  blocks.  A  single  factor  repeated-measures  Analysis  of  Variance  (AN OVA) 
indicated  that  the  trial  Block  had  a  significant  effect  on  subjects’  mean  hit  rate  [A(9,207)  =  4.38, 
MSe  =  0.0 1 1 ,  /?  <  .01].  To  better  understand  how  hit  rate  varied  across  blocks,  a  series  of  post- 
hoc  comparisons  were  performed  to  contrast  each  block  with  every  other  block.  These  post-hoc 
comparisons  were  performed  using  Fisher’s  Least  Significant  Difference  (LSD)  method,  which 
computes  an  LSD  value  for  each  pair-wise  comparison  of  means  and  determines  the  probability 
that  the  difference  was  the  result  of  random  chance.  The  results  of  the  comparisons  are 
summarized  in  Table  5. 
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Figure  3.  Hit  rates  (correct  engagement  of  enemy)  across  all  blocks  of  trials. 

The  baseline  and  end  baseline  hit  rates  did  not  differ  significantly,  indicating  that  there  was  no 
learning  effect  across  the  course  of  the  experimental  session.  Because  there  was  no  variability  in 
the  features  of  friend  and  enemy  BOTs  in  the  baseline  conditions,  they  indicated  the  expected 
level  of  performance  when  subjects  are  certain  of  the  distinguishing  features  of  friends  and 
enemies.  By  comparing  the  hit  rates  of  the  baselines  to  the  hit  rates  of  all  other  conditions  (in 

DRDC  Toronto  TR  2009-127  1  1 


Table  5.  Fisher’s  Least  Significant  Difference  (LSD)  Test  Results  (Hit  Rate) 


Baseline 

Uncertain  Visual  Cue 

Uncertain  Behavioural  Cue 

End 

Baseline 

Uncertain  Friend 

Uncertain  Enemy 

Uncertain  Friend 

Uncertain  Enemy 

FSVV 

FNSVV 

ESVV 

ENSVV 

FSBV 

FNSBV 

ESBV 

ENSBV 

Baseline 

N/A 

0.780 

0.328 

0.070 

0.163 

0.008 

0.163 

0.328 

0.210 

0.094 

FSVV 

- 

N/A 

0.484 

0.037 

0.094 

0.018 

0.094 

0.484 

0.328 

0.163 

FNSVV 

- 

- 

N/A 

0.006 

0.018 

0.094 

0.018 

1.000 

0.780 

0.484 

ESVV 

- 

- 

- 

N/A 

0.675 

<0.001 

0.674 

0.006 

0.002 

0.001 

ENSVV 

- 

- 

- 

- 

N/A 

<0.001 

1.000 

0.018 

0.008 

0.002 

FSBV 

- 

- 

- 

- 

- 

N/A 

<0.001 

0.094 

0.163 

0.328 

FNSBV 

- 

- 

- 

- 

- 

- 

N/A 

0.018 

0.008 

0.002 

ESBV 

- 

- 

- 

- 

- 

- 

- 

N/A 

0.780 

0.484 

ENSBV 

- 

- 

- 

- 

- 

- 

- 

- 

N/A 

0.675 

End 

Baseline 

- 

- 

- 

- 

- 

- 

- 

- 

- 

N/A 

Within  MSe  =  0.011;  df  =  207 

Contrasts  which  are  significant  to  p<.05  are  shown  in  bold  italic 
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which  uncertainty  was  associated  with  a  feature  of  the  friend  or  enemy  BOTs),  we  can 
determine  whether  or  not,  and  to  what  extent,  uncertainty  concerning  the  distinguishing 
features  of  friend  and  enemy  BOTs  affected  subjects  performance. 

The  mean  hit  rate  was  lower  in  several  blocks  than  that  of  the  baseline  (ESVV,  ENSVV 
FNSBV)  but  these  differences  were  not  statistically  significant.  Only  the  FSBV  block 
exhibited  a  significantly  higher  hit  rate  than  the  baseline.  Hit  rates  in  the  ESVV,  ENSVV,  and 
FNSBV  blocks  were  significantly  lower  than  that  of  the  end  baseline.  This  finding,  in 
conjunction  with  the  non-significant  differences  of  these  blocks  from  the  baseline  (no 
uncertainty),  suggests  that  hit  rate  was  reduced  by  variability  of  the  enemy  BOTs’  salient  and 
non-salient  visual  features  and  the  friendly  BOTs’  non-salient  behavioural  feature. 

Uncertainty  of  the  characteristics  of  friends  (i.e.,  cases  in  which  friends  may  possess 
characteristics  associated  with  enemies)  did  not  reduce  hit  rate,  except  in  one  block,  even 
though  one  might  have  expected  subjects  to  be  less  willing  to  engage  enemies  because  of  the 
increased  risk  of  misclassifying  a  friend  as  an  enemy.  Subjects  either  had  no  difficulty 
recognizing  friends  despite  cue  uncertainty  or  were  applying  strict  decision  criteria  that 
discouraged  engaging  any  target  perceived  to  have  a  friendly  characteristic. 
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Figure  4.  False  Alarm  rates  (incorrect  engagement  of  friend)  across  all  blocks  of  trials. 


False  Alarm  (FA)  Rate 

Mean  FA  rates  (see  Figure  4)  ranged  between  15-25%,  which  is  much  higher  than  any 
commander  would  wish  to  observe  in  an  operational  environment.  For  the  purpose  of  this 
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experiment,  however,  it  was  essential  that  subjects  made  FAs  at  a  suitably  high  rate  to  allow 
statistical  discrimination  of  differences  between  blocks. 

Although  FA  rates  were  somewhat  higher  in  the  FSVV,  ENSVV,  FSBV,  and  ENSBV  blocks 
than  the  baseline  block,  a  single  factor  repeated-measures  ANOVA  revealed  no  significant 
overall  effect  of  trial  Block  on  subjects’  mean  FA  rate  [F(9,207)  =  1.39,  MSe  =  0.015,  n.s.]. 
Thus,  these  differences  are  not  statistically  reliable  and  no  post  hoc  comparisons  were  made. 

Sensitivity 

Signal  Detection  Theory  (SDT)  [16]  is  a  way  to  examine  subjects’  sensitivity  to  stimuli.  In 
the  context  of  the  CID  task,  sensitivity  refers  to  subjects’  psychological  discrimination 
between  friends  and  foes,  or  their  ability  to  correctly  classify  a  friend  as  a  friend,  and  a  foe  as 
a  foe.  As  a  statistical  measure,  sensitivity  (d1)  is  defined  in  terms  of  z,  the  inverse  of  the 
normal  function  and  the  observed  hit  (H)  and  FA  rates  [17]  by  the  formula: 

d'  =  z(H)  -  z(FA)  (1) 


Block 

Figure  5.  Mean  d’  Values  (Sensitivity)  across  all  blocks  of  trials. 

The  z-transformation  converts  the  H  and  FA  rates  to  standard  deviation  units  such  that  a  d' 
value  of  zero  corresponds  to  a  complete  inability  to  distinguish  friend  from  foe.  Increasingly 
positive  values  of  d'  indicate  progressively  greater  ability  to  discriminate  friend  from  foe. 
Sensitivity  takes  into  account  correct  engagements  of  enemy  BOTs  and  correct  non- 
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engagements  of  friendly  BOTs  and  is  thus  a  more  complete  measure  of  performance  than  H 
rate  or  FA  rate  alone. 

Figure  5  shows  mean  d'  scores  for  each  block.  A  single  factor  repeated-measures  ANOVA 
revealed  no  significant  overall  effect  of  trial  block  on  d'  [F(9,207)  =  1.39,  MSe  =  0.015,  n.s.]. 
Thus,  no  post  hoc  comparisons  were  made. 

Response  Bias 

Derived  from  SDT,  response  bias  is  a  measure  of  a  subject’s  general  tendency  to  respond 
positively  to  a  target.  In  the  case  of  the  current  experiment,  it  measures  the  general  tendency 
to  report  a  foe  regardless  of  the  actual  identity  of  the  target. 

Like  sensitivity  (d1),  response  bias  (c)  is  defined  in  terms  of  z  and  the  observed  H  and  FA  rates 
[17]: 

c  =  -0.5  •  [z(H)  +  z(FA)\  (2) 

Positive  values  of  c  indicate  a  tendency  to  classify  a  target  as  foe  regardless  of  its  true 
identity,  whereas  negative  values  indicate  a  tendency  to  classify  a  target  as  a  friend.  The 
larger  the  value  in  either  direction,  the  greater  the  tendency. 


Block 

Figure  6.  Mean  c  Values  (Response  Bias)  across  all  blocks  of  trials. 

Figure  6  shows  the  mean  c  scores  calculated  for  each  block.  A  single  factor  repeated- 
measures  ANOVA  revealed  a  significant  effect  of  trial  block  [A( 9,207)  =  3.27,  MSe  =  0.432, 
p  <  .01],  To  better  understand  how  response  bias  varied  across  blocks,  a  series  of  post-hoc 
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Table  6.  Fisher’s  Least  Significant  Difference  (LSD)  Test  Results  (Response  Bias) 


Baseline 

Uncertain  Visual  Cue 

Uncertain  Behavioural  Cue 

End 

Baseline 

Uncertain  Friend 

Uncertain  Enemy 

Uncertain  Friend 

Uncertain  Enemy 

FSW 

FNSW 

ESVV 

ENSW 

FSBV 

FNSBV 

ESBV 

ENSBV 

Baseline 

N/A 

0.222 

0.269 

0.100 

0.295 

0.063 

0.205 

0.490 

0.126 

0.466 

FSW 

- 

N/A 

0.908 

0.004 

0.024 

0.521 

0.013 

0.057 

0.754 

0.622 

FNSW 

- 

- 

N/A 

0.006 

0.032 

0.449 

0.018 

0.073 

0.668 

0.705 

ESVV 

- 

- 

- 

N/A 

0.547 

<0.001 

0.704 

0.338 

0.002 

0.018 

ENSW 

- 

- 

- 

- 

N/A 

0.004 

0.824 

0.722 

0.010 

0.077 

FSBV 

- 

- 

- 

- 

- 

N/A 

0.002 

0.011 

0.742 

0.256 

FNSBV 

- 

- 

- 

- 

- 

- 

N/A 

0.563 

0.005 

0.047 

ESBV 

- 

- 

- 

- 

- 

- 

- 

N/A 

0.027 

0.156 

ENSBV 

- 

- 

- 

- 

- 

- 

- 

- 

N/A 

0.420 

End 

Baseline 

- 

- 

- 

- 

- 

- 

- 

- 

- 

N/A 

Within  MSe  =  0.432;  df  =  207 

Contrasts  which  are  significant  to  p<.05  are  shown  in  bold  italic 
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comparisons  was  performed  using  Fisher’s  LSD  method.  The  results  of  the  comparisons  are 
summarized  in  Table  6. 

As  shown  in  Figure  6,  subjects  exhibited  small  negative  biases  in  the  Baseline  and  End 
Baseline  blocks,  indicating  a  general  tendency  to  not  engage  targets.  Biases  in  all  other 
blocks  did  not  differ  significantly  from  the  Baseline  value.  Only  the  biases  in  the  ESVV  and 
FNSBV  blocks  differed  significantly  from  the  End  Baseline.  Thus,  the  overall  differences 
among  blocks  do  not  seem  to  constitute  very  large  shifts  in  response  bias. 

Nevertheless,  we  see  that  subjects  exhibited  positive  bias  in  the  ESVV,  ENSVV,  FNSBV,  and 
ESBV  blocks.  This  suggests  that  when  enemy  BOTs  possessed  uncertain  characteristics, 
subjects  exhibited  a  slight  bias  to  engage,  although  the  c  values  are  small.  Positive  bias, 
however,  is  not  seen  in  the  ENSBV  block,  perhaps  because  the  variable  behavioural 
characteristic  was  non-salient.  Conversely,  subjects  exhibited  negative  bias  in  the  FSVV, 
FNSVV,  FSBV,  and  ENSBV  blocks,  suggesting  subjects  exhibited  a  general  tendency  to  not 
engage  targets  when  friendly  BOTs  possessed  uncertain  characteristics.  Negative  bias, 
however,  was  not  observed  in  the  FNSBV  block,  perhaps  because  the  variable  behavioural 
characteristic  was  non-salient. 
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Discussion 


Both  hit  rate  and  FA  rate  are  important  indicators  of  performance  in  this  experimental  task 
and,  indeed,  real-world  combat  where  soldiers  must  successfully  but  safely  engage  enemy 
forces.  Reductions  in  hit  rates  correspond  to  decreased  combat  effectiveness,  which  opens 
possibilities  for  the  enemy  to  survive  and  cause  harm  to  one’s  own  forces.  Flowever, 
increased  FA  rates  correspond  to  instances  of  fratricide  and  direct  harm  to  own  forces  being 
caused  by  own  forces.  Although  the  results  of  this  experiment  showed  only  very  small  effects 
on  hit  rate  and  response  bias,  it  remains  important  to  consider  how  the  variability  of 
perceptual  and  behavioural  cues  might  affect  CID  in  the  field. 

Subjects’  hit  rates  were  affected  by  uncertainty  associated  with  visual  and  behavioural 
characteristics  of  targets  in  the  environment.  This  effect  depended  on  the  salience  of  the 
characteristics  that  were  uncertain.  Presumably  because  less  salient  characteristics  are  less 
likely  to  be  considered  in  the  CID  decision  [16],  subjects’  hit  rates  were  not  affected  in  the 
FNSBV  block.  Although  hit  rates  in  the  FSVV  and  FNSVV  blocks  were  very  similar  to  those 
of  both  the  baseline  and  end  baseline,  the  hit  rate  of  the  FSBV  block  significantly  exceeded 
those  of  both  baseline  blocks.  In  contrast,  the  hit  rate  of  the  FNSBV  block  was  lower  than 
those  of  both  baseline  blocks.  Although  we  had  expected  that  hit  rate  could  decrease  as  a 
result  of  characteristic  uncertainty,  it  is  unclear  why  such  an  effect  would  appear  only  when 
non-salient  behavioural  characteristics  were  uncertain. 

Subjects’  FA  rates  were  not  affected  by  cue  uncertainty,  although  FA  rates  were  somewhat 
higher  than  baseline  in  the  FSVV,  ENSVV,  FSBV,  and  ENSBV  blocks.  It  may  be  that 
stronger  manipulations  of  cue  uncertainty  could  produce  increased  FA  rates. 

Subjects’  sensitivity  was  also  not  affected  by  cue  uncertainty,  indicating  that  they  remained 
equally  good  at  distinguishing  friend  and  foe  despite  variations  in  the  characteristics  of  friends 
and  foes.  Subjects  did,  however,  exhibit  small  differences  in  response  bias  across  blocks. 

This  result  suggests  that  subjects  shifted  their  decision  criterion  in  response  to  cue  uncertainty 
associated  with  targets.  Subjects  appear  to  have  slightly  lowered  their  criterion  when  enemies 
appeared  with  uncertain  cues  but  more  dramatically  raised  their  criterion  when  friendly  BOTs 
appeared  with  uncertain  cues.  These  shifts  in  criteria  indicate  that  subjects  were  sensitive  to 
changes  in  the  certainty  with  which  cues  predicted  a  target’s  classification.  Thus,  when  some 
enemy  BOTs  appeared  with  a  feature  associated  with  friendly  BOTs,  subjects  lowered  their 
criterion  to  be  able  to  correctly  engage  enemies.  This  shift  in  criterion,  however,  did  not  have 
an  appreciable  effect  on  hit  rate  or  sensitivity.  In  contrast,  when  friendly  BOTs  appeared  with 
a  feature  associated  with  enemies,  subjects  raised  the  criterion  to  avoid  the  error  of  engaging  a 
friend.  This  change  in  criterion  presumably  contributed  to  the  lower  hit  rates  seen  in  those 
blocks  in  which  friendly  BOTs  possessed  uncertain  cues. 

The  nature  of  modem  warfare  makes  uncertainty  a  pressing  issue.  Soldiers  can  expect  to  find 
themselves  operating  in  environments  in  which  friendly,  enemy,  and  neutral  factions  employ 
similar  or  identical  equipment,  wear  similar  or  identical  clothing,  and  potentially  employ 
similar  tactics  (e.g.,  [7]).  Hostile  factions  in  asymmetric  conflicts  can  be  expected  to 
purposely  mimic  civilians  in  order  to  confuse  soldiers.  An  enemy,  by  adopting  characteristics 
of  friendly  forces  or  characteristics  that  are  readily  mistakable  for  those  of  friendly  forces,  can 
decrease  own  force  effectiveness.  The  art  of  disguise  has  of  course  been  known  from  the 
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advent  of  warfare  but  it  takes  on  much  greater  significance  in  today’s  asymmetric  conflicts 
with  the  addition  of  large  numbers  of  neutral/civilian  entities  likely  to  be  in  close  proximity. 

In  addition,  the  CF  often  works  in  coalitions,  which  can  undermine  soldiers’  familiarity  with 
the  appearance  of  friendly  units.  Friendly  forces  will  not  intentionally  take  on  characteristics 
of  the  enemy.  Nevertheless,  it  is  possible  to  unintentionally  take  on  those  characteristics  or, 
more  likely,  characteristics  that  can  be  misperceived  as  those  of  the  enemy.  This  is  especially 
true  in  coalition  environments  where  own  forces  will  be  comprised  of  soldiers  using  different 
equipment  and  trained  in  different  procedures.  Of  course,  when  enemies  attempt  to  blend  into 
neutral/civilian  populations  they  are  attempting  to  create  confusion  concerning  what 
characteristics  reliably  signal  enemy  presence. 

The  risk  posed  by  cue  uncertainty  can  be  mitigated  in  several  ways.  TTPs  can  be  revised  to 
help  soldiers  recognize  the  potential  for  the  visual  and  behavioural  characteristics  of  targets  to 
be  imperfect  predictors  of  identity.  Training  and  education  can  prepare  soldiers  to  anticipate 
and  recognize  that  equipment  and  clothing  can  be  similar  among  factions.  A  related  approach 
is  to  develop  SOPs  based  on  operational  experience  to  eliminate  the  use  of  cues  that  have 
proved  unreliable. 

Decision  support  systems  can  also  help.  Although  SA  support  systems,  such  as  Battlefield 
Target  Identification  Device  (BTID),  do  not  directly  aid  in  target  identification,  improved 
spatial  awareness  of  friendly  units  can  reduce  confusion  caused  by  visual  and  behavioural 
uncertainty  in  targets.  More  direct  measures  to  support  identification,  such  as  the  use  of 
Radio  Frequency  (RF)  tags,  provide  soldiers  with  a  tool  to  reliably  identify  friendly  units. 

Finally,  the  findings  of  this  experiment  will  contribute  to  the  ongoing  effort  to  develop  a 
cognitive  model  of  CID  decision  making  and  information  aggregation  [18].  A  cognitive 
model  is  needed  to  complement  procedural  models  such  as  INCIDER  [3]  and  provide  a  means 
to  predict  the  impact  of  operational  factors  on  target  identification  performance.  This  model 
will  also  be  useful  in  assessing  the  impact  of  decision  support  concepts  before  expensive 
technology  development  begins. 
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